ROC surface assessment of the ANB angle and Wits appraisal's diagnostic performance with a statistically derived 'gold standard': does normalizing measurements have any merit?

Objective To assess the ANB angle's and Wits appraisal's diagnostic performance using an extended version of Receiver Operating Curve (ROC) analysis, which renders ROC surfaces. These were calculated for both the conventional and normalized cephalometric tests (calculated by exchanging the patient's reference landmarks with those of the Procrustes superimposed sample mean shape).The required 'gold standard' was derived statistically, by applying generalized Procrustes superimposition (GPS) and principal component analysis (PCA) to the digitized landmarks, and ordering patients based upon their PC2 scores. Methods Digitized landmarks of 200 lateral cephalograms (107 males, mean age: 12.8 years, SD: 2.2, 93 females, mean age: 13.2 years, SD: 1.7) were subjected to GPS and PCA. Upon calculating the conventional and normalized ANB and Wits values, ROC surfaces were constructed by varying not just the cephalometric test's cut-off value within each ROC curve, but also the gold standard cut-off value over different ROC curves in 220 steps between -2 and 2 standard deviations along PC2. The volume under the resulting ROC surfaces (VUS) served as a measure of overall diagnostic performance. The statistical significance of the volume differences was determined using permutation tests (1000 rounds, with replacement). Results The diagnostic performance of the conventional ANB and Wits was remarkably similar for both Class I/II (81.1 and 80.75% VUS, respectively, P > 0.05). Normalizing the measurements improved all VUS highly significantly (91 and 87.2 per cent, respectively, P < 0.001). Conclusion The conventional ANB and Wits do not differ in their diagnostic performance. Normalizing the measurements does seem to have some merit.